The present invention relates to spectral methods in general and, more particularly, to spectral imaging methods for cell classification, biological research, medical diagnostics and therapy, which methods are referred to hereinbelow as spectral bio-imaging methods. The spectral bio-imaging methods of the present invention can be used to provide automatic and/or semiautomatic spectrally resolved morphometric classification (e.g., detection, grading) of neoplasm, by standardizing the measured spectra of biological material present in biological samples in general, but more specifically when these samples are stained for microscopy imaging, and as a result enabling the building of libraries of spectral signatures for these biological objects, and therefore allowing automatic and semiautomatic analysis of such samples. Also, cytological or tissue section specimen which are generally stained with chromogenic dyes, simultaneously with, or without, specific tagged or marked expressed proteins and/or genes and/or DNA segments, to be measured by bright field light microscopy have the advantages over fluorescently dyed, tagged and/or marked specimen, of being significantly more permanent and therefore of providing means to repeat the tests at later times, of avoiding the background signals due to auto fluorescence of the specimen itself, and in general of being less expensive, at least at present. By helping overcome the signal variations due to drifts in staining colors and concentrations, due to different manufacturing processes of the stains, different origins of the stains, and other environmental reasons, and instabilities of the measuring instrumentation, due to drifts in the illumination spectrum and/or intensity, changes in the optics spectral transmission, and spectral response of the detector array, the methods can be used for the improvement and automatization of the detection of the spatial organization and of the qualification and quantification of cellular and tissue constituents and structures associated with, for example, tumorogenesis, using, for example, light transmission microscopy combined with high spatial and spectral resolutions. Furthermore, the methods of the present invention can be used to improve the detection of cellular spatial organization and the quantification of cellular and tissue natural constituents, domains and structures, including, but not limited to, proteins, genes, DNA sections, subcellular organelles, and the like, using light transmission, reflection, scattering and fluorescence emission strategies, with high spatial and spectral resolutions, and may therefore be employed for classification of cancer cells and/or grading and/or staging the progression of cancer, using, what is referred herein as, spectrally resolved morphometry, mainly for diagnostic and prognostic applications. In particular the methods of the present invention can be used for classification of cells to developmental stages, and to qualify and quantify metabolic processes within cells. The methods can further be used to develop new and more fine tuned indexes for neoplasm classification (including grading), which will eventually replace the existing indexes. Although the explanations and the treatment is shown for full spectral measurements encompassing a large number of wavelengths, it will be recognized on the basis of the description of the method, the assumptions and the provided mathematical modeling, that the method of the present invention is useful and valid for spectral imaging measurements which contain an arbitrary number of wavelengths in the defined spectral range, from one single wavelength to hundreds (the usual maximum number used in this technology), and more.
A spectrometer is an apparatus designed to accept light, to separate (disperse) it into its component wavelengths, and measure the lights spectrum, that is the intensity of the light as a function of its wavelength. An imaging spectrometer is a spectrometer which collects incident light from a scene and measures the spectra of each pixel (i.e., picture element) thereof.
Spectroscopy is a well known analytical tool which has been used for decades in science and industry to characterize materials and processes based on the spectral signatures of chemical constituents therein. The physical basis of spectroscopy is the interaction of light with matter. Traditionally, spectroscopy is the measurement of the light intensity emitted, scattered or reflected from or transmitted through a sample, as a function of wavelength, at high spectral resolution, but without any spatial information.
Spectral imaging, on the other hand, is a combination of high resolution spectroscopy and high resolution imaging (i.e., spatial information). Most of the works so far described concern either obtaining high spatial resolution information from a biological sample, yet providing only limited spectral information, for example, when high spatial resolution imaging is performed with one or several discrete band-pass filters [See, Andersson-Engels et al. (1990) Proceedings of SPIExe2x80x94Bioimaging and Two-Dimensional Spectroscopy, 1205, pp. 179-189], or alternatively, obtaining high spectral resolution (e.g., a full spectrum), yet limited in spatial resolution to a small number of points of the sample or averaged over the whole sample [See for example, U.S. Pat. No. 4,930,516, to Alfano et al.].
Conceptually, a spectral bio-imaging system consists of (i) a measurement system, and (ii) an analysis software. The measurement system includes all of the optics, electronics and the manner in which the sample is illuminated (e.g., light source selection), the mode of measurement (e.g., fluorescence or transmission), as well as the calibration best suited for extracting the desired results from the measurement. The analysis software includes all of the software and mathematical algorithms necessary to analyze and display important results in a meaningful way.
Spectral imaging has been used for decades in the area of remote sensing to provide important insights in the study of Earth and other planets by identifying characteristic spectral absorption features originating therefrom. However, the high cost, size and configuration of remote sensing spectral imaging systems (e.g., Landsat, AVIRIS) has limited their use to air and satellite-born applications [See, Maymon and Neeck (1988) Proceedings of SPIExe2x80x94Recent Advances in Sensors, Radiometry and Data Processing for Remote Sensing, 924, pp. 10-22; Dozier (1988) Proceedings of SPIExe2x80x94Recent Advances in Sensors, Radiometry and Data Processing for Remote Sensing, 924, pp. 23-30].
There are three basic types of spectral dispersion methods that might be considered for a spectral bio-imaging system: (i) spectral grating or prism, (ii) spectral filters and (iii) interferometric spectroscopy. As will be described below, the latter is best suited to implement the method of the present invention, yet as will be appreciated by one ordinarily skilled in the art, grating, prism and filters based spectral bio-imaging systems may also be found useful in some applications.
In a grating or prism (i.e., monochromator) based systems, also known as slit-type imaging spectrometers, such as for example the DILOR system: [see, Valisa et al. (September 1995) presentation at the SPIE Conference European Medical Optics Week, BiOS Europe 1995, Barcelona, Spain], only one axis of a CCD (charge coupled device) array detector (the spatial axis) provides real imagery data, while a second (spectral) axis is used for sampling the intensity of the light which is dispersed by the grating or prism as function of wavelength. The system also has a slit in a first focal plane, limiting the field of view at any given time to a line of pixels. Therefore, a full image can only be obtained after scanning the grating (or prism) or the incoming beam in a direction parallel to the spectral axis of the CCD in a method known in the literature as line scanning. The inability to visualize the two-dimensional image before the whole measurement is completed, makes it impossible to choose, prior to making the measurement, a desired region of interest from within the field of view and/or to optimize the system focus, exposure time, etc. Grating and prism based spectral imagers are in use for remote sensing applications, because an airplane (or satellite) flying over the surface of the Earth provides the system with a natural line scanning mechanism.
It should be further noted that slit-type imaging spectrometers have a major disadvantage since most of the pixels of one frame are not measured at any given time, even though the fore-optics of the instrument actually collects incident light from all of them simultaneously. The result is that either a relatively large measurement time is required to obtain the necessary information with a given signal-to-noise ratio, or the signal-to-noise ratio (sensitivity) is substantially reduced for a given measurement time. Furthermore, slit-type spectral imagers require line scanning to collect the necessary information for the whole scene, which may introduce inaccuracies to the results thus obtained.
Filter based spectral dispersion methods can be further categorized into discrete filters and tunable filters. In these types of imaging spectrometers the spectral image is built by filtering the radiation for all the pixels of the scene simultaneously at a different wavelength at a time by inserting in succession narrow band filters in the optical path, or by electronically scanning the bands using acousto-optic tunable filters (AOTF) or liquid-crystal tunable filter (LCTF), see below. Similarly to the slit type imaging spectrometers equipped with a grating or prism as described above, while using filter based spectral dispersion methods, most of the radiation is rejected at any given time. In fact, the measurement of the whole image at a specific wavelength is possible because all the photons outside the instantaneous wavelength being measured are rejected and do not reach the CCD.
Tunable filters, such as AOTFs and LCTFs have no moving parts and can be tuned to any particular wavelength in the spectral range of the device in which they are implemented. One advantage of using tunable filters as a dispersion method for spectral imaging is their random wavelength access; i.e., the ability to measure the intensity of an image at a number of wavelengths, in any desired sequence without the use of filter wheels. However, AOTFs and LCTFs have the disadvantages of (i) limited spectral range (typically, xcexmax=2xcexmin) while all other radiation that falls outside of this spectral range must be blocked, (ii) temperature sensitivity, (iii) poor transmission, (iv) polarization sensitivity, and (v) in the case of AOTFs an effect of shifting the image during wavelength scanning, demanding careful and complicated registration procedures thereafter.
All these types of filter and tunable filter based systems have not been used successfully and extensively over the years in spectral imaging for any application, because of their limitations in spectral resolution, low sensitivity, and lack of easy-to-use and sophisticated software algorithms for interpretation and display of the data.
A method and apparatus for spectral analysis of images which have advantages in the above respects was disclosed in U.S. patent application Ser. No. 08/392,019 to Cabib et al., filed Feb. 21st, 1995, now U.S. Pat. No. 5,539,517, issued Jul. 23, 1996, which is incorporated by reference as if fully set forth herein, with the objective to provide a method and apparatus for spectral analysis of images which better utilizes all the information available from the collected incident light of the image to substantially decrease the required frame time and/or to substantially increase the signal-to-noise ratio, as compared to the conventional slit- or filter type imaging spectrometer, and does not involve line scanning. According to this invention, there is provided a method of analyzing an optical image of a scene to determine the spectral intensity of each pixel thereof by collecting incident light from the scene; passing the light through an interferometer which outputs modulated light corresponding to a predetermined set of linear combinations of the spectral intensity of the light emitted from each pixel; focusing the light outputted from the interferometer on a detector array, scanning the optical path difference (OPD) generated in the interferometer for all pixels independently and simultaneously and processing the outputs of the detector array (the interferograms of all pixels separately) to determine the spectral intensity of each pixel thereof. This method may be practiced by is utilizing various types of interferometers wherein the OPD is varied to build the interferograms by moving the entire interferometer, an element within the interferometer, or the angle of incidence of the incoming radiation. In all of these cases, when the scanner completes one scan of the interferometer, the interferograms for all pixels of the scene are completed.
Apparatuses in accordance with the above features differ from the conventional slit- and filter type imaging spectrometers by utilizing an interferometer as described above, therefore not limiting the collected energy with an aperture or slit or limiting the incoming wavelength with narrow band interference or tunable filters, thereby substantially increasing the total throughput of the system. Thus, interferometer based apparatuses better utilize all the information available from the incident light of the scene to be analyzed, thereby substantially decreasing the measurement time and/or substantially increasing the signal-to-noise ratio (i.e., sensitivity). The sensitivity advantage that interferometric spectroscopy has over the filter and grating or prism methods is known in the art as the multiplex or Fellgett advantage [see, Chamberlain (1979) The principles of interferometric spectroscopy, John Wiley and Sons, pp. 16-18 and p. 263].
Consider, for example, the xe2x80x9cwhisk broomxe2x80x9d design described in John B. Wellman (1987) Imaging Spectrometers for Terrestrial and Planetary Remote Sensing, SPIE Proceedings, Vol. 750, p. 140. Let n be the number of detectors in the linear array, mxc3x97m the number of pixels in a frame and T the frame time. The total time spent on each pixel in one frame summed over all the detectors of the array is nT/m2. By using the same size array and the same frame rate in a method according to the invention described in U.S. Pat. No. 5,539,517, the total time spent summed over all the detectors on a particular pixel is the same, nT/m2. However, whereas in the conventional grating or prism method the energy seen by every detector at any time is of the order of 1/n of the total, because the wavelength resolution is 1/n of the range, in a method according to the invention described in U.S. patent application Ser. No. 08/392,019 the energy is of the order of unity, because the modulating function is an oscillating function (e.g., sinusoidal (Michelson) or similar periodic function such as low finesse Airy function with Fabry-Perot) whose average over a large OPD range is 50%. Based on the standard treatment of the Fellgett advantage (or multiplex advantage) described in interferometry textbooks [for example, see, Chamberlain (1979) The principles of interferometric spectroscopy, John Wiley and Sons, pp. 16-18 and p. 263], it is possible to show that devices according to this invention have measurement signal-to-noise ratios which are improved by a factor of n0.5 in the cases of noise limitations in which the noise level is independent of signal (system or background noise limited situations) and by the square root of the ratio of the signal at a particular wavelength to the average signal in the spectral range, at wavelengths of a narrow peak in the cases the limitation is due to signal photon noise. Thus, according to the invention described in U.S. Pat. No. 5,539,517, all the required OPDs are scanned simultaneously for all the pixels of the scene in order to obtain all the information required to reconstruct the spectrum, so that the spectral information is collected simultaneously with the imaging information. This invention can be used with many different optical configurations, such as a telescope for remote sensing, a microscope for laboratory analysis, fundus cameras for retinal imaging, fiber optics and endoscopes for industrial monitoring and medical imaging, diagnosis, therapy and others.
In a continuation application (U.S. patent application Ser. No. 08/571,047 to Cabib et al., filed Dec. 12, 1995, now U.S. Pat. No. 5,784,162, issued Jul. 21, 1998, which is incorporated by reference as if fully set forth herein) the objective was to provide spectral imaging methods for biological research, medical diagnostics and therapy, which methods can be used to detect spatial organization (i.e., distribution) and to quantify cellular and tissue natural constituents, structures, organelles and administered components such as tagging probes (e.g., fluorescent probes) and drugs using light transmission, reflection, scattering and fluorescence emission strategies, with high spatial and spectral resolutions. In U.S. Pat. No. 5,784,162 the use of the spectral imaging apparatus described in U.S. Pat. No. 5,539,517 for interphase fluorescent in situ hybridization of as much as six loci specific probes (each loci located on a different chromosome) was demonstrated, as well as additional biological and medical applications.
Spectral bio-imaging systems are potentially useful in all applications in which subtle spectral differences exist between chemical constituents whose spatial distribution and organization within an image are of interest. The measurement can be carried out using virtually any optical system attached to the system described in U.S. Pat. No. 5,539,517, for example, an upright or inverted microscope, a fluorescence microscope, a macro lens, an endoscope and a fundus camera. Furthermore, any standard experimental method can be used, including light transmission (bright field and dark field), auto-fluorescence and fluorescence of administered probes, etc.
Fluorescence measurements can be made with any standard filter cube (consisting of a barrier filter, excitation filter and a dichroic mirror), or any customized filter cube for special applications, provided that the emission spectra fall within the spectral range of the system sensitivity. Spectral bio-imaging can also be used in conjunction with any standard spatial filtering method such as dark field and phase contrast, and even with polarized light microscopy. The effects on spectral information when using such methods must, of course, be understood to correctly interpret the measured spectral images.
U.S. patent application Ser. No. 08/824,234, filed Mar. 25, 1997, which is incorporated by reference as if fully set forth herein teaches methods for automatic and/or semiautomatic spectrally resolved morphometric classification (e.g., detection, grading) of neoplasm, which are designed to provide objective, as opposed to subjective cell (e.g., cancer cell) classification. According to the method disclosed therein (a) a sample including at least a portion of at least one cell is prepared to be spectrally imaged; (b) the sample is viewed through an optical device optically connected to an imaging spectrometer for obtaining a spectrum of each pixel of the sample; (c) each of the pixels is classified into classification groups according to the pixels spectra; and (d) by analyzing the classification groups of pixels, the cells of the sample are classified into cell classes. This method was exemplified with respect to breast carcinomas.
Morphometric classification of various tumor types, such as, but not limited to, leukemias, lymphomas, sarcomas and other carcinomas [see, for example, Clarke A M, Reid W A and Jack A S (1993) Combined proliferating cell nuclear antigen and morphometric analysis in the diagnosis of cancerous lymphoid infiltrates. J. Clin. Pathol. 46:129-134] are vastly implemented both in research medical practice.
Nevertheless, as was recently published following an NIH workshop which evaluated the reliability of histopathological diagnosis by the best pathologists in the field of cancer diagnostics, there is a discordance among expert pathologists in the diagnosis of neoplasm. Based on this workshop, it was concluded that histopathological decision making is 100% subjective, regardless of the origin of specimen and that this state of affairs in histopathological diagnosis is not confined to a specific tumor, but is applicable to differential diagnosis in every organ. These conclusions were published in an editorial by A Bernard Ackerman (1996) entitled xe2x80x9cDiscordance among expert pathologists in diagnosis of melanocytic neoplasmxe2x80x9d, in Human pathology 27:1115-1116.
Characterization of nuclear features by different techniques is used for determination of diagnosis, treatment and prognosis. Quantitative estimation of various histopathological parameters such as two dimensional estimates of nuclear profile area, nuclear profile densities and mitotic profile numbers have been shown to correlate with differentiation and prognosis. Alterations in nuclear structure are the morphologic hallmark of cancer diagnosis. Nuclear size, shape, chromatin pattern have all been reported to change in various cancer types. For example, blood cancer cells (e.g., leukemia and lymphoma cells) are unique in their morphological features, which render them amenable for detection and classification via morphometric analysis. The following provides few examples. B-ALL is characterized by lymphoblasts having a small cytoplasm, lacking a nucleolus and having an open chromatin structure. CLL is characterized by small yet apparently mature lymphocytes having a small cytoplasm and condensed chromatin structure. PLL is characterized by lymphocytes having a large cytoplasm, round nucleus and pronounced nucleolus. HCL is characterized by lymphocytes featuring cytoplasmatic projections, giving them a xe2x80x9chairyxe2x80x9d appearance. T-ALL is characterized by lymphoblasts having a cleaved or convoluted nucleus and open chromatin structure. Sezary syndrome is characterized by small or large cells featuring cerebriformed nucleus and/or vacuoles and condensed chromatin structure. Additional examples and descriptions associated with these and other diseases and available in Foon K A, Todd R F. Immunologic classification of leukemia and lymphoma. Blood 68:1, 1986; The Non-Hodgkin""s lymphoma pathologic classification project: NCI sponsored study of classification of NHL""s Summary and description of working formulation for clinical uses. Cancer 49:2112, 1982; A revised European American classification of lymphoid neoplasms: A proposal from the International Lymphoma Study Group. Harris N J et al. Blood 84:1361, 1994, which are incorporated by reference as if fully set forth herein.
All these methods, however, employ only image (i.e., spatial) information for analysis. A whole new dimension of analysis may be added using spectral information which reflects the interaction between light and matter. The combination of both spatial and spectral information will largely contribute to cancer detection and classification.
There is thus a widely recognized need for, and it would be highly advantageous to have, spectral bio-imaging methods and spectral morphometric methods for cells classification devoid of the above described limitations, especially the subjectivity of pathologists in neoplasm diagnosis, which provide advanced and quantitative, semi or fully automatic, means for cancer classification. In addition there a widely recognized need for, and it would be highly advantageous to have, spectral bio-imaging method which accounts for day-to-day variation associated with most of the presently employed cell staining protocols.
According to the present invention there are provided spectral bio-imaging methods which can be used for automatic and/or semiautomatic spectrally resolved morphometric classification (e.g., grading) of cells (e.g., neoplasm).
The methods can also be used for classification of cells into grades, developmental stages, and to qualify and quantify metabolic processes within cells. The method can further be used to develop new and more fine tuned indexes for neoplasm and embryonic cells classification.
According to further features in preferred embodiments of the invention described below, there is provided a method of spectral-morphometric analysis of biological samples, the biological samples including substantially constant components and suspected variable components, the method comprising the steps of (a) using a spectral data collection device for collecting spectral data of picture elements of the biological samples; (b) defining a spectral vector associated with picture elements representing a constant component of at least one of the biological samples; (c) using the spectral vector for defining a correcting function being selected such that when operated on spectral vectors associated with picture elements representing other constant components, spectral vectors of the other constant components are modified to substantially resemble the spectral vector; (d) operating the correcting function on spectral vectors associated with at least the variable components for obtaining corrected spectral vectors thereof; and (e) classifying the corrected spectral vectors into classification groups.
According to still further features in the described preferred embodiments the method further comprising the step of (f) presenting pixels associated with each of the classification groups in a distinctive color.
According to still further features in the described preferred embodiments the substantially constant components are red blood cells.
According to still further features in the described preferred embodiments the red blood cells are added to the biological sample.
According to still further features in the described preferred embodiments the red blood cells are inherent to the biological sample.
According to still further features in the described preferred embodiments the suspected variable components are tumor cells, tumor tissues or parts thereof.
According to still further features in the described preferred embodiments the tumor cells are hematopoietic tumor cells.
According to still further features in the described preferred embodiments the suspected variable components are cells infected by a pathogen
According to still further features in the described preferred embodiments the biological sample is a blood sample of a patient suspected to have or having a hematopoietic tumor.
According to still further features in the described preferred embodiments the hematopoietic tumor is selected from the group consisting of leukemia and lymphoma.
According to still further features in the described preferred embodiments the biological sample is of a patient suspected of having or having a disease selected from the group consisting of ALL, CLL, IM, PCL, PLL and Sezary syndrome.
According to still further features in the described preferred embodiments prior to collecting spectral data of picture elements of the biological samples, the biological sample is stained.
According to still further features in the described preferred embodiments staining the biological sample is effected via a stain selected from the group consisting of an immunohistochemical stain, a histological stain, a DNA ploidy stain, a nucleic acid sequence specific probe and any combination thereof.
According to still further features in the described preferred embodiments the histological stain is selected from the group consisting of Hematoxylin-Eosin stain, May Grunwald Giemsa stain, Romanowsky Giemsa, Masson""s trichrome and Papanicolaou stain.
According to still further features in the described preferred embodiments collecting spectral data of picture elements of the biological samples is effected by (i) collecting incident light simultaneously from all pixels of the sample using collimating optics; (ii) passing the incident collimated light through an interferometer system having a number of elements, so that the light is first split into two coherent beams which travel in different directions inside the interferometer and then the two coherent beams recombine to interfere with each other to form an exiting light beam; (iii) passing the exiting light beam through a focusing optical system which focuses the exiting light beam on a detector having a two-dimensional array of detector elements, so that at each instant each of the detector elements is the image of one and always the same pixel of the sample for the entire duration of the measurement, so that the real image of the sample is stationary on the plane of the detector array and at any time during the measurement the image is still visible and recognizable, and so that each of the detector elements produces a signal which is a particular linear combination of light intensity emitted by the pixel at different wavelengths, wherein the linear combination is a function of the instantaneous optical path difference; (iv) scanning one or more of the elements of the interferometer system, so that the optical path difference between the two coherent beams generated by the interferometer system is scanned simultaneously for all the pixels of the sample; and (v) recording signals of each of the detector elements as function of time using a recording device to form a spectral cube of data.
According to still further features in the described preferred embodiments collecting spectral data of picture elements of the biological samples is effected by (i) illuminating the sample with broadband light through a step-scan interferometer and condenser; (ii) collecting and optically imaging transmitted or reflected light through the sample onto a two-dimensional array of detector elements (such as a CCD or focal plane array); (iii) recording a number of frames from all the pixels of the sample in synchronization with each step-scanned position of the interferometer; (iv) mathematically processing each interferogram function so obtained for each pixel of the image to obtain a spectrum as function of wavelength for each of said pixels; and optionally, (v) performing spectral correction, classification or color display of the pixels based on their so obtained and corrected spectral vectors, as described herein.
According to still further features in the described preferred embodiments the spectral data collection device includes an element selected from the group consisting of a dispersion element, a filter and an interferometer.
According to still further features in the described preferred embodiments classifying the corrected spectral vectors into classification groups is effected using a classification map algorithm which employs reference spectral vectors for associating picture elements into the classification groups.
According to still further features in the described preferred embodiments the reference spectral vectors for classification are of a previously prepared reference library.
According to still further features in the described preferred embodiments at least one of the reference spectral vectors for classification is of picture elements derived from a cell domain selected from the group consisting of nucleolus, inter-chromosomal region, cytoplasm, a first chromatin region of the nucleus, a second chromatin region of the nucleus and background.
According to still further features in the described preferred embodiments the spectral vector is a normalized spectral vector.
According to still further features in the described preferred embodiments classifying the corrected spectral vectors into classification groups is effected by spectral vector maxima classification.
The present invention successfully addresses the shortcomings of the presently known configurations by providing a method for automatic or semi automatic and internally referenced method for classification and grading of neoplasm. Furthermore, the method of the present invention provides spectrally resolved morphometric classification images which may be used by pathologists for classification and grading of neoplasm, which images replace the prior art RGB images and lead to more objective interpretation of the results and therefore, to a more accurate classification. This in turn may affect diagnosis, treatment and prognosis.